Hybrid marker-less camera pose tracking with integrated sensor fusion

This paper proposes an algorithm for visual-inertialcamera pose tracking, using adaptive recursive particle filtering.The method benefits from the agility of inertial-based androbustness of vision-based tracking. A proposal distribution hasbeen developed for the selection of the particles, which takes intoaccount the characteristics of the Inertial Measurement Unit(IMU) and the motion kinematics of the moving camera. A set ofstate-space equations are formulated, particles are selected andthen evaluated using the corresponding features tracked byoptical flow. The system state is estimated using the weightedparticles through an iterative sequential importance resamplingalgorithm. For the particle assessment, epipolar geometry, andthe characteristics of focus of expansion (FoE) are considered. Inthe proposed system the computational cost is reduced byexcluding the rotation matrix from the process of recursive stateestimations. This system implements an intelligent decisionmaking process, which decides on the best source of trackingwhether IMU only, hybrid only or hybrid with past statecorrection. The results show a stable tracking performance withan average location error of a few centimeters in 3D space. Keywords— motion tracking, camera pose tracking, 6DOF,Inertial, IMU, particle filtering, optical flow, focus of expansion,SLAM, PTAM,INTRODUCTIONA. MotivationCamera pose tracking is an assisting technology inenabling the accurate and continuous recovery of the six degreeof freedom (6DOF) position and orientation of a movingcamera, with the most prominent challenges in real-timesystems. The potential applications of accurate 6DoF posetracking are numerous, including entertainment and immersivegames, augmented reality, industrial maintenance andengineering, architecture, medicine, assisted living for theelderly, security, education, prototyping and autonomousnavigation systems. Proposals and solutions for pose recovery have been somany in the past few decades. Among all, visual SLAM andPTAM based solutions provided reasonable accuracyespecially for the Augmented Reality (AR) applications;however they were mostly reported to be limited in wide areatracking measurements and uncontrolled real-time localizationdue to the expensive computational cost involved [1], [2].Moreover, in recent years the hybrid systems consisting oflow cost inertial measurement units (IMUs) and the robust andhigh-dimensional computer vision-aided algorithms haveenhanced the performance and agility of the tracking systems[3], [4], [5]. Such solutions have also tackled the hurdles ofreal-time sensing and localization especially in GPS-deniedenvironments [6], [7].Inertial-Visual Pose Tracking usingOptical Flow-aided Particle Filtering